Moments and Deviations

We have bounded the expected running times of several randomized algorithms in first two chapters. While the expectation of a random variable (such as a running time) may be small, it may frequently assume values that are far higher. In analyzing the performance of a randomized algorithm, we often like to show that the behavior of the algorithm is good almost all the time. For example, it is more desirable to show that the running time is small with high probability, not just that it has a small expectation. The similar statement: for randomized algorithms, usually knowing the bound of expected running time is not enough. It is more desirable to show that the expected running time is small with high probability. To prove this statement, we will begin by examining a family of stochastic processes that is fundamental to the analysis of many types of randomized algorithms. They are Occupancy Problems. This motivates the study of general bounds on the probability that a random variable deviates far from its expectation, enabling us to avoid such custom-made analysis. The probability that a random variable deviates by a given amount from its expectation is referred to as a tail probability for that deviation.